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ABSTRACT 

Evaluation findings concerning six Austin Independent 
School District (AISD) Assessment Centers are presented. The report 
describes the AISD assessment center, summarizes the evaluation 
findings, and provides conclusions and recommendations. The AISD 
assessment center is one' of seven components which make up the 
District's Administrative Leadership program. The other components 
are early identification, recruitment, screening, development, 
selection and placement, and evaluation. Issues addressed in this 
interim report include: size of pool of unassessed applicants, 
ethnicity of assessment center participants, new appointments and 
promotions among assessment center participants available in the 
administrative candidate bank, and the relationship between 
assessment center scores and administrator perceiver scores. Results 
indicated the size of the pool of unassessed applicants continued to 
grow. The ethnic distribution of participants was in agreement with 
the goals of the Faculty/Staff Recruitment Plan. Assessment center 
participants* success rate was high. More high scoring candidates 
were appointed, or promoted, than low scoring. A few low Lcoiing 
assessment center participants were appointed, or promoted, to new 
administrative positions. A low correlation was found between 
assessment center scores and administrator perceiver scores. (DWH) 
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EXECUTIVE SUMMARY 



AISD Assessment Center: 1983-84 
Interim Report for the First Six Assessment Centers 

Evaluation findings concerning the AISD Assessment Center are presented in 
the 1983-84 Interim Report. The issues addressed in this report are: (1) 
Size of pool of unassessed applicants, (2) Ethnicity of Assessment Center 
participants, (3) Success rate of Assessment Center participants, (4) New 
appointments and promotions among Assessment Ceiater participants, (5) Assess- 
ment Center participants available in administrative candidate bank, 
(6) Relationship between Assessment Center scores and administrator perceiver 
scores. The major findings are summarized below. 



• The size of the pool of unassessed applicants concinued to grow. 

• The ethnic distribution of the Assessment Center participants 
was in agreement with the goals of the District's Faculty /Staff 
Recruitment Plan. 

• The success rate of the Assessment Center participants was high. 

• The success rate of l:he Black participants was much lower than 
that of the Hispanic and Anglo/Other participants. The success 
rate of the male participants was lower than that of the female 
participants. 

• There were ,more high-scoring candidates being appointed or pro- 
moted than there were low-scoring candidates. 

• A rew low-scoring Assessmetit Center participants were appointed 
or promote^ to new administrative positions. 

• The number of successful participants being placed represented 
26% of the total number of participants who successfully com- 
pleted the six Assessment Centers and 31% of the total promotions 
and new appointments. 

• Thirty-five participants who have succaaAfully completed the Assess- 
ment Center are available in the administrative candidate bank. 



• A low correlation was found between the Assessment Center scores 
and the administrator perceiver scores. 



S»iiranary page from the Office of Research and Evaluation Report on the AISD 
Asse;,sment Center, ORE * )lication Number 83.38. 
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INTERIM REPORT 



Prolect Title ; AISD Assessment Center: 1983-84 Interim Report 

for the First SiT. Assessment Centers 

C ontact Persons ; Glynn Ligon, Maria Ramos-Cancel Wicker 

Decision Question: Should AISD continue to use Assessment Centers in 

selection or promotion of administrators? 



Major Positive Findings ; 

1. The overall ethnicity distribution df the Assessment Center partici- 
pants (Hispanic, 30%; Black, 17%; Anglo/Other, 53%) was in agreement 
with the goals the District's Faculty/Staff Recruitment Plan. 

2. In general, the success rate of the Assessment Center participants 
was high (75% for Assessment Centers 1, 2, and 6; 67% for Assess- 
ment Centers 4 and 5; 5b% for Assessment Center 3). 

3. Examination of the participants being placed reveals that there were 
more high-scoring candidates being appointed or promoted than there 
were low-scoring candidates (13 vs. 3) . • 

4. Thirty-five participants who successfully completed the Assessment 
Center and have not been already placed are available in the admin- 
istrative candidate bank. This represents a source for future 
appointments and promotions. 

Major Findings Requiring Action ; 

1. While the number of new applicants decreased somewhat with subsequent 
Assessment Centers, the size of (:he pool of unassessed applicants 
continued to grow (from 70 after Assessment Center 2 to 214 after 
Assessment Center 6) . 

2. The success rate of the Black participants was much lower than 
that of the Hispanic and Anglo/Other participants (41% vs. 68% 
and 79%, respectively). The success rate of the male participants 
was lower than that of the female participants (59% vs. 75%). 
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3. Examination of the participants being placed reveals that high- 
scoring candidates were not the only ones receiving new appoint- 
ments or promotions. Also, the number of successful participants 
being placed represented 26% of the total number of participants 
who successfully completed the six Assessment Centers and 31% of 
the total promotions and new appointments. This indicates that 
greater emphasis was placed on information not obtained from the 
Assessment Center and/or placement decisions v^ere made in response 
to hiring considerations agreed to by the DisLrict. 

A. A low correlation (.24) was found between the Assessment Center scores 
and the ad^nistrator perceiver scores obtained by the participants. 
This indicates that, overall, the administrator perceiver scores may 
not be a very good predictor of Assessment Center performance. 

5. The relationship between the Assessment Center scores and on-the-job 
performance after new placement could not be determined for the 
present report, primarily because the latter measures have not yet 
been completed. This relationship should be examined after the on- 
the-job performance measures are completed. 

These are areas that need to be examined more carefully in future evalua- 
tions if AISD wishes to continue using Assessment Centers in the selection 
or promotion of administrators. 



Recommendation ; 

Based on the present evaluation tindings, it seems advisable for AISD to 
fulfill its thtee-year commitment to the Assessment Center process. The 
remaining part of this time period may allow the District to strengthen 
those areas which have proven to be successful and to work on those that 
seem to require more attention. This time period may also allow the 
District to gather other information useful in deciding whether to con- 
tinue using the Assessment Center after the three-year commitment has 
been fulfilled. Also, the possibility of conducting a validation study, 
similar to the one performed by NASSP, may be considered. 
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INTRODUCTION 

This Is the second Interim Report for the AISD Assessment Center. The 
first report, completed in January, 1983, was for the first two Assess- 
ment Centers • The present report is for all six centers, including the 
first two. The first part of this report gives a brief description of 
the AISD Assessment Center. The second part sunnnarizes the evaluation 
findings, and the third provides some conclusions and recommendations. 

DESCRIPTION OF THE AISD ASSESSMENT CENTER 

The AISD Assessment Center is one of seven components which comprise the 
District's Administrative Leadership Program developed for the purpose 
of staffing administrative positions with the best qualified individuals. 
The other components are: Early Identification, Recruitment, Screening, 
Development, Selection and Placement, and Evaluation. These are illus- 
trated in Figure 1. 
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Figure 1. COMPONENTS OF THE DISTRICT'S ADMINISTRATIVE LEADERSHIP PROGRAM. 
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The Assessment Center is a two-day event during which twelve candidates 
respond to assigned tasks that rely heavily on simulation techniques. 
The tisks are designed to assess administrative skills in twelve identified 
areas listed in Figure 2. Candidates are observed by six trained assessors 
who spend the remainder of the week in study, consultation, and writing. 
The result is a recommendation and a written report on each candidate. 
This report is shared with the candidate in an individual conference with 
the center director usually within a week to ten days after the completion 
of the Assessment Center^ The report is confidential and is available only 
to the candidate and to District administrators responsible for filling 
vacant positions. 



Figure 2. ASSESSMENT CENTER ADMINISTRATIVE SKILLS AREAS. 

The twelve candidates who participate in each Assessment Center are selected 
through the screening component of the Administrative Leadership Program. 
The projected needs of the District are taken into account in the selection 
process, as well as the District's Affirmative Faculty/Staff Recruitment 
Plan. The administrative profile and report resulting from the Assessment 
Center is intendad to serve as ,a basis for making staff selection and place- 
ment decisions. Candidat2S who successfully complete the Assessment Center 
have their files placed in the District's administrative candidate bank, 
where they will be considered for any vacancy for which they are qualified. 
In addition, they will have the opportunity to participate in the develop- 
ment component of the Administrative Leadership Program. 

Successful completion of an Assessment Center is defined as having received 
from assessors an overall consensus rating of three or higher. The rating 
scale is from one to five, with five being the highest (l«poor, 2»below 
average, 3«average, 4"above average, 5*outstanding) . The overall consen- 
sus rating represents the overall level of performance of the candidate 
as judged by the assessors* 



Skill Areas 
Given Priority 



Other Skill Areas Assessed 



Problem Analysis 
Judgement 
Sensitivity 
Leadership 



Organizational Ability 
Decisiveness 
Stress Tolerance 
Educational Values 



Oral Communication 
Written Communication 
Personal Motivation 
Range of Interest 
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The AISD Assessment Center Is a part of the National Association of Secondary 
School Principals (NASfJP) Assessment. Project, which began In 1975 after 
extensive research and development In coop. ration with the American Psycho- 
logical Association. The Assessment Center model, which has been succesful 
In business and Industry, was adapted to the educational setting to assess 
the administrative skills needed by elementary and secondary building level 
administrators. This model has received the support of the National Ele- 
mentary Principals and Supervisors Association. A summary of the findings 
from the NASSP Assessment Center validation study can be found In the 
1982-83 Interim Report of the first two AISD Assessment Centers (ORE Publi- 
cation No. 82.40). This report also Includes Information pertaining to cost 
and to the Initial implementation efforts to ensure that the Center is imple- 
mented In the manner Intended by NASSP. 

EVALUATION FINDINGS FOR THE AISD ASSESSMENT CENTER 

Six Assessment Centers were offered from March, 1982 to November, 1983. 
Application to participate in the Assessment Center is open to persons 
seeking their first administrative positions or a promotion in the division 
of instruction. The applications are periodically screened to determine 
which applicants will go through the Administrator Percelver Interview 
developed by Selection Research Incorporated (SRI) . Assessment Center 
participants are selected from those interviewed. Applicants not selected 
to participate in the Assessment Center may remain in the pool for possi- 
ble selection at a later date. 

Application and Selection Process - Pool of Unassessed Applicants 



Figure 3 summarizes the application and selection process in terms of the 
total number of applicants, the number of applicants interviewed, and the 
niimber of applicants assessed. The figure shows that 286 people applied 
for the six Assessment Centers, the first having the largest number of 
applicants. Forty-four percent (125) of the total applicants were inter- 
viewed, and of these 58% (72) were selected to participate in the Assess- 
ment Centers. While the number of new applicants has decreased somewhat 
with subsequent Assesstnent Centers, the size of the pool of unassessed 
applicants continues to grow. 

Ethnicity and Sex of Applicants 

Figure 4 gives a breakdown of the applicants according to ethnicity and 
sex. Of the 72 applicants assessed, 30% were Hispanic, 17% Black, and 
53% Anglo or Other. These percentages are in agreement with, the goals 
of the District's Faculty/Staff Recruitment Plan. With respect to sex. 
Figure 4 shows that the male/female percentages for the applicants assessed 
are not in line with the male/female percentages for the total applicants 
(applicants assessed: 38% male/62% female; total applicants: 52% male/ 
48% female). However, the percentages for the applicants assessed are con- 
sistent with the District's effort to maintain female representation in 
administrative positions. 
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*Ciunulatlve total of applicants Interviewed divided by cumulative total of all applicants. 
.■**Cuir':latlve total of applicants assessed divided by cumulative total of applicants Interviewed. 
***New applicants assessed divided by size of pool of all unassessed applicants before AC. 
****New applicants assessed divided by size of pool of applicants Interviewed before AC. 



Figure 3. NUMBER OF APPLICANTS AND SIZE OF POOL OF UNASSESSED APPLICANTS 
FOR EACH CENTER. 
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APPLICANTS 

(N) 



ETHNICITY 



HISPANIC BLACK ANGLO/OTHER 

■n (%) N (%) N (%) 



SEX 



FEMALE MAI.E 
N % N % 



APPLICANTS 
ASSESSED 
(72) 



22 (30) 12 (17) 38 (53) 



45 (62) 27 (38) 



APPLICAFTS 
INTERVIEWED 
(125) 



28 (22) 16 (13) 81 (65) 



76 (61) 49 (39) 



APPLICANTS 

NOT 
INTERVIEWED 
(161) 



34 (21) 29 (18) 98 (61) 



TOTAL 

APPLICANTS 62 (22) 45 (16) 179 (62) 
(286) 



62 (39) 99 (61) 



138. (48) 148 (52) 



Figure 4. NUMBER OF APPLICANTS BY ETHNICITY AND SEX.* 



Ethnicity and Sex of Participants for each Assessment Center 

Figure 5 gives a breakdown of the participants according to center, ethnicity, 
and sex. This figure shows that although the overall percentages for ethnicity 
were in agreement with the goals of the District's Faculty/Staff Recruitment 
Plan, representation of the three ethnic groups varied among centers. However, 
it would be difficult, and possibly unrealistic, to attempt to fit the goals 
to each Assessment Center. 



*As a goal, the District attempts to recruit and employ minorities in the 
following percentages: Black (15-20%), Hispanic (25-30%), Anglo/Other 
(50-55%) , It attempts also to place male and female representation at 
all levels of employment whenever possible. 
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Figure 5. NUMBER OF PARTICIPANTS BY CENTER, ETHNICITY AND SEX. 



Adminls tr ator Percelver Scores of Assessment Center Participants 

Applicants who were Interviewed received an administrator percelver score. 
The highest possible score was 60 and the lowest, zero. Figure 6 shows the 
mean score received by the group of applicants who later "participated In the 
Assessment Center. This score Is broken down by ethnicity and sex. The 
group of Anglo/Other participants obtained the highest mean score compared 
to the Hispanic anu Black whose mean scores differed only slightly from 
each other. There was a wide range of scores In all ethnic groups, with the 
Anglo/Other group having the highest minimum-maximum scores. The mean score 
for the female group was higher than the mean score for the male group. The 
female group had higher minimum-maximum scores. The wide range of scores 
indicate that the administrator percelver score was not the most Important 
factor in selecting the Assessment Center participants. Also, the differ- 
ence in niniraum-maximum scores among groups indicates that the criteria 
might have varied somewhat depending on other factors, such as ethnicity 
and sex. 
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ETHNICITY 



SEX 



Hispanic Black Anglo/Other Female Male 



Mean 30.05 29.50 

Range 29 17 

(Min.-Max.)* (20-49) (23-40) 



34.71 
28 
(27-55) 



33.36 



30.85 



31 23 

(24-55) . (20-44) 



'''Lowest and highest scores 

Figure 6. MEAN AND RANGE OF ADMINISTRATOR PERCEIVER SCORES BY 

ETHNICITY AND SEX "FOR THE INTERVIEWED ASSESSMENT CENTER 
PARTICIPANTS. 



Success Rate of Assessment Center Participants 

In general, the success rate of the Assessment Center participants was high. 
Figure 7 shows that 69% of the 72 participants successfully completed the 
Assessment Center. Higher ratings were obtained in Assessment Centers 1, 
2, and 6, where 75% of the participants received an overall consensus rating 
of three or higher. The group mean rating is highest for the second Assess- 
ment Center and lowest for the third. 
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Figure 7. RATINGS RECEIVED BY PARTICIPANTS, SUCCESSFUL 
COMPLETION, AND GROUP MEAN RATINGS IN THE 
SIX ASSESSMENT CENTERS. 
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The success rate was high for Anglo/Other and Hispanic participants. How- 
ever, this is not the case for the Black participants. Figure 8 shows that 
79% of the Anglo /Others and 68% of the Hispanics received a rating of three 
or higher, whereas 41% of the Blacks received the same rating. Successful 
completion was higher among the female participants. The group mean rating 
was highest for the Anglo/Other and female participants. 
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Figure 8. RATINGS RECEIVED BY PARTICIPANTS, SUCCESSFUL COMPLETION, 
AND GROUP MEAN RATINGS BY ETHNICITY AND SEX. 



New A ppointments and Promotions Among Assessment Center Participants 

Since July, 1982, 51 new administrative appointments or promotions for 
principals, assistant principals, and supervisors/coordinators have been 
made. Thiis figure does not include transfers within the same position to 
a new location. Sixteen of the participants in the first six Assessment 
Centers have received new appointments or promotions and two have changed 
locations. Figure 9 shows the Assessment Center ratings of those who were 
promoted or newly appointed. Thirteen of those 16 participants had succesr- 
fully completed the Assessment Center. The fact that this figure represents 
only 26% of the total number of participants who successfully completed 
the six centers and only 31% of the total promotions and new appointments 
indicates that the Assessment Center rating has not been the most important 
information in the consideration of a candidate for administrative positions. 
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Figure 9. RATINGS OF ALL ASSESSMENT CENTER PARTICIPANTS AND 
OF PARTICIPANTS PROMOTED OR NEWLY APPOINTED. 



Figure 10 gives a breakdown by ethnicity, sex and rating of the participants 
who were promoted or newly appointed. The figure shows chat 50?-; of the par- 
ticipants promoted or newly appointed were Anglo/Other. Also, there were 
more Hispanic than Black and more females than males. 
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Figure 10. NUMBER OF PARTICIPANTS BY ETHNICITY, SEX, AND 
RATING WHO WERE PROMOTED OR NEWLY APPOINTED. 



Figure 11 shows the level of Interest of the participants who were promoted 
or newly appointed. Forty-four percent of these participants had expressed 
interest in an administrative position at the elementary level. Thirty-one 
percent had expressed interest in an administrative position at the second- 
ary level. The rest were either interested in a position as coordinator/ 
supervisor or in more than one administrative position. 
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LEVEL 


OF INTEREST* 








PARTICIPANTS 


E 


s 


c 


ES 


EC 


sc 


ESC 


TOTAL 


All AssesBment 
Center 
Participants 


24 

(33%) 


26 

(36%) 


4 

(6%) 


4 

(6%) 


8 

(ll'O 


3 

(4%) 


3 

(4%) 


72 


Successful 
Participants 


17 

(34%) 


20 

(40%) 


3 

(6%) 


2 

(4%) 


5 

(10%) 


1 

(2%) 


2 

(4%) 


50 


Participants 
Promoted or 
Newly Appointed 


7 

(44%) 


5 

(31%) 


1 

(6%) 


0 

(0%) 


0 

(0%) 


1 

(6%) 


1 

_(6%) 


16 



*E-Elementary, S-Secondary, C-Coordinator/Supervisor; ES-Elementary or Second- 
ary, EC-Elementary or Coordinator/ Supervisor , SC»Secondary or Coordinator/ 
Supervisor; ESC»Elementary, Secondary, or Coordinator /Supervisor . 



Figure 11. 



LEVEL OF INTEREST OF ALL ASSESSMENT CENTER PARTICIPANTS, 
OF SUCCESSFUL PAPTICIPANTS , AND OF PARTICIPANTS PROMOTED 
OR NEWLY APPOINTED. 



Assessment Center Participants Available in Administrative Candidate Bank 

Of the 50 participants who successfully completed the Assessment Center, 35 
are still available in the administrative candidate bank. Figure 12 gives 
a breakdown of these participants according to center. The number of suc- 
cessful participants not available is also reported. 



ASS. 


SUCCESSFUL 


NOT 


AVAILABLE 




CENTER 


COMPLETION 


P/A 


W 


TOTAL 


AVAILABLE 


1 


9 


A 


0 


4 


5 


2 


9 


2 


1 


3 


6 


3 


7 


1 


1 


2 


5 


4 


8 


3 


0 


3 


5 


.'j 


8 


3 


0 


3 


5 


6 


9 


0 


0 


0 


9 


Total 


50 


13 


2 


15 


35 



P/A: Persons who recently 
received a promotion 
or a new assignment 

W: Persons who resigned 
from the District or 
ai*» currently on leave 



Figure 12. PARTICIPANTS WHO SUCCESSFULLY COMPLETED THE SIX 
ASSESSMENT CENTERS AND ARE STILL AVAILABLE IN 
THE ADMINISTRATIVE CANDIDATE BANK. 
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Correlation Between Assessment Center Scores 
and the Administrator Percelver Scores 



The Assessment Center scores were correlated with the administrator percelver 
scores obtained by the participants. A low correlation (•24) was found be- 
tween these two sets of scores. This result indicates uhaC, overall > the 
administrator percelver score may not be a very good predictor of Assess- 
ment Center performance. Separate correlation analyses performed for the 
different ethnic and sex groups ^ showed a moderate correlation (•AA) between 
the two sets of scores for the female group. However, no significant corre- 
lation was found for any of the other groups. This information is summarized 
in Figure 13. 









E.THNICITY 




SEX 




Overall 


Hispanic 


Black 


Other 


Female Male 


Correlation 


.24 


.31 


-.08 


.14 


.44 -.09 


Level of 
Significance 


.02 


.08 


.40 


.20 


.001 .33 



Figure 13. CORRELATIONS BETWEEN ASSESSMENT CENTER SCORES 
AND THE ADMINISTRATOR PERCEIVER SCORES • 



Figure 14 gives the administrator percelver mean scores for the participants 
according to the scores received in the Assessment Center. Although the 
participants with the highest Assessment Center scores tended to have a 
slightly higher administrator percelver mean score than the participants 
with lower Assessment Center scores, the difference was not statistically 
significant. Figure 14 shows that there was a wide range of administra- 
tor percelver scores among the three groups of Assessment Center scores. 
There were participants who had obtained low administrator percelver scores 
among all three groups. 



ACS 


1 - 2 


3 


4-5 


Mean 


30.86 


32.06 


34.94 


Range 


16 


24 


34 


(Min.-Max.)* 


(23-39) 


(20-44) 


(21-55) 



*Lovest and highest scores 



Figure 1.4. MEAN AND RANGE OF ADMINISTRATOR PERCEIVER 
SCORES BY ASSESSMENT CENTER SCORES. 
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CONCLUSIONS AND RECOMMENDATIONS 

Based on the present evaluation findings, It seems advisable for AISD to 
fulfill Its three-year commitment to the Assessment Center process. The 
remaining part of this time period may allow the District to strengthen 
those areas which have proven to be successful and to work on those that 
seem to require more attention. This time period may also allow the 
District to gather other information useful in deciding whether to con- 
tinue using the Assessment Center after the three-year commitment has 
been fulfilled. Also, the possibility of conducting a validation study, 
similar to the one performed by NASSP, may be 'considered. 

Because the Assessment Center is one of seven components that comprise the 
Administrative Leadership Program, which in turn forms part of the larger 
AISD community, it seems important that future evaluations be conducted 
within this broader framework. This may provide a better understandix:<? of 
the Assessment Center process and its effects upon other divisions and 
programs within the District, such as the Faculty/Staff Recruitment Plan, 
the Division of Instruction, the Administrative Leadership Program as a 
whole, and the Division of Operations in general. 
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